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Abstract. Probabilistic automata (PA) [22] have been successfully applied in the formal 
verification of concurrent and stochastic systems. Efficient model checking algorithms have 
been studied, where the most often used logics for expressing properties are based on PCTL 
and its extension PCTL* [J. Various behavioral equivalences are proposed for PAs, as 
a powerful tool for abstraction and compositional minimization for PAs. Unfortunately, the 
behavioral equivalences are well-known to be strictly stronger than the logical equivalences 
induced by PCTL or PCTL*. This paper introduces novel notions of strong bisimulation 
relations, which characterizes PCTL and PCTL* exactly. We extend weak bisimulations 
characterizing PCTL and PCTL* without next operator, respectively. Further, we also 
extend the framework to simulations. Thus, our paper bridges the gap between logical 
and behavioral equivalences in this setting. 



1. Introduction 

Probabilistic automata (PA) [22] have been successfully applied in the formal verification of 
concurrent and stochastic systems. Efficient model checking algorithms have been studied, 
where properties are mostly expressed in the logic PCTL, introduced in [11] for Markov 
chains, and later extended in [1] for Markov decision processes, where PCTL is also extended 
to PCTL*. 

To combat the infamous state space problem in model checking, various behavioral 
equivalences, including strong and weak bisimulations, are proposed for PAs. Indeed, they 
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Figure 1: Counter example of strong probabilistic bisimulation. 

turn out to be a powerful tool for abstraction for PAs, since bisimilar states implies that they 
satisfy exactly the same PCTL formulae. Thus, bisimilar states can be grouped together, 
allowing one to construct smaller quotient automata before analyzing the model. More- 
over, the nice compositional theory for PAs is exploited for compositional minimization [5], 
namely minimizing the automata before composing the components together. 

For Markov chains, i.e., PAs without nondeterministic choices, the logical equivalence 
implies also bisimilarity, as shown in [3] . Unfortunately, it does not hold in general, namely 
PCTL equivalence is strictly coarser than bisimulation - and their extension probabilistic 
bisimulation - for PAs. Even there is such a gap between behavior and logical equivalences, 
bisimulation based minimization is extensively studied in the literatures to leverage the 
state space explosion, for instance see [6| fll fl6] . 

The main reason for the gap can be illustrated by the following example. Consider the 
PAs in FigfT] where assuming that s\,S2,S3 are three absorbing states with different state 
properties. It is easy to see that s and r are PCTL equivalent: the additional middle transi- 
tion out of r does not change the extreme probabilities. Existing bisimulations differentiate 
s and r, mainly because the middle transition out of r cannot be matched by any transi- 
tion (or combined transition) of s. Bisimulation requires that the complete distribution of 
a transition must be matched, which is in this case too strong, as it differentiates states 
satisfying the same PCTL formulae. 

In this paper we will bridge this gap. We introduce novel notions of behavioral equiva- 
lences which characterize (both soundly and completely) PCTL, PCTL* and their sublogics. 
Summarizing, our contributions are: 

• A new bisimulation characterizing PCTL* soundly and completely. The bisimu- 
lation arises from a converging sequence of equivalence relations, each of which 
characterizes bounded PCTL*. 

• Branching bisimulations which correspond to PCTL and bounded PCTL equiva- 
lences. 

• We then extend our definitions to weak bisimulations, which characterize sublogics 
of PCTL and PCTL* with only unbounded path formulae. 

• Further, we extend the framework to simulations as well as their characterizations 
as well. 

Organization of the paper. Section [2] introduces some notations. In Section [3] we recall def- 
initions of probabilistic automata, bisimulation relations by Segala |21] . We also recall the 
logic PCTL* and its sublogics. Section |4] introduces the novel strong and strong branching 



BISIMULATIONS MEET PCTL EQUIVALENCES FOR PROBABILISTIC AUTOMATA 



3 



bisimulations, and proves that they agree with PCTL* and PCTL equivalences, respec- 
tively. Section [5] extends them to weak (branching) bisimulations, and Section [6] extends 
the framework to simulations. We discuss the coarsest congruent bisimulations and simu- 
lations in Section and the extension to countable states in Section In Section [9] we 
discuss related work, and Section \W\ concludes the paper. 

2. Preliminaries 

Probability space. A (discrete) probability space is a tuple V = (Cl,F,rj) where Cl is a 
countable set, F = 2 n is the power set, and r] : F — > [0, 1] is a probability function which 
is countable additive. We skip F whenever convenient. Given probability spaces {Vi = 
(i^i,r]i)}i e j and weights W{ > for each i such that Yli£i w i = 1> the convex combination 
Yliei w (Pi is defined as the probability space (Cl, rj) such that Cl = Cli and for each set 
YCCl, r ] (Y) = Y; ieI w l r H (Yr)Cl i ). 

Distributions. We denote by Dist(S) the set of discrete probability spaces over S. We shall 
use s,r, t, ... and [x,v... to range over S and Dist(S), respectively. The support of fi is 
defined by supp(fi) := {s G S \ fi(s) > 0}. For an equivalence relation TZ, we write fi TZ v if 
it holds that /i(C) = v(C) for all equivalence classes C G S/1Z. A distribution \i is called 
Dirac if \supp(fi)\ = 1, and we let V s denote the Dirac distribution with V s (s) = 1. 

Downward Closure. Below we define the downward closure of a subset of states. 

Definition 1. For pre-order 1Z over S and CCS, define C-ji = {s' \ s' 1Z s A s G C}. We 

say C is TZ downward closed iff C = (Jr . 

We use Sfc as the shorthand of {s}n, and TZ = {Cjz \ C C S} denotes the set of all TZ 
downward closed sets. 

3. Probabilistic Automaton, PCTL* and Bisimulations 

Definition 2. A probabilistic automator^ is a tuple V = (S,—>,IS,AP,L) where S is a 
finite set of states, ->CSx Dist(S) is a transition relation, IS C S is a set of initial states, 
AP is a set of atomic propositions, and L : 5 — > 2 AP is a labeling function. 

As usual we only consider image-finite PAs, i.e. {(r, /i) G— >-| r = s} is finite for each 
s E S. A transition (s, /x) G— >■ is denoted by s — > jjL. Moreover, we write fj, — > fi' iff for each 
s G supp(fi) there exists s fj, s such that //(r) = ^ /i(s) • /U s (r). 

A pai/i is a finite or infinite sequence oj = S0S1S2 ■ • • of states. For each i > there exists 
a distribution fj, such that Sj — )• ^ and ^(s^+i) > 0. We use lstate(uj) and 1(uj) to denote the 
last state of oj and the length of oj respectively if oj is finite. The sets Path is the set of all 
paths, and Path(so) are those starting from so- Similarly, Path* is the set of finite paths, 
and Path*(so) are those starting from sq. Also we use oj[i] to denote the (i + l)-th state for 
i > 0, wj* to denote the fragment of oj ending at oj[i], and oj\i to denote the fragment of oj 
starting from oj[i]. 



In this paper we omit the set of actions, since they do not appear in the logic PCTL we shall consider 
later. Note that the bisimulation we shall introduce later can be extended to PA with actions directly. 
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We introduce the definition of scheduler to resolve nondeterminism. A scheduler is a 
function a : Path* — > Dist{-^) such that a(uj)(s,fi) > implies s = lstate{ui). A scheduler 
a is deterministic if it returns only Dirac distributions, that is, the next step is chosen 
deterministically. We use 

Path(s ,a) = {ui G Path(s ) | Vi > 0.3/^.cr(a;| i )(a;[i], fi) > 0A/j(w[i + 1]) > 0} 

to denote the set of paths starting from so respecting a. Similarly, Path* (sq, a) only contains 
finite paths. 

The cone of a finite path uj, denoted by C u , is the set of paths having ui as their prefix, 
i.e., = {uj' I uj < uj'} where uj' < uj iff uj' is a prefix of uj. Fixing a starting state sq and a 
scheduler a, the measure Prob a ^ so of a cone C w , where ui = sqS\ . . . Sk, is defined inductively 
as follows: Prob a S0 (C U ) equals 1 if k = 0, and for k > 0, 

Prob atSQ (Cu) = Prob a , So (C u \k-i) • ^ <r(a;| A!_1 )(s fc _i, //) • fi'(s k ) 

\(s fc _iy)e-> 

Let be the smallest algebra that contains all the cones and is closed under complement 
and countable unions. Prob a)So can be extended to a unique measure on B. 

Given a pre-order 1Z over 5, is the set of 1Z downward closed paths of length i 
composed of 7Z downward closed sets, and is equal to the Cartesian product of 7Z with 
itself i times. Let 1Z* = Ui>i]V be the set of 1Z downward closed paths of arbitrary length. 
Define l(Q) = n for G 7Z n . For = CqC\ . . . C n G ]Z*, the 7£ downward closed cone Cq, 
is defined as Cq, = {C^ \ ui G 0}, where uj G iff € C, for < i < n. 

For distributions \x\ and /i2, we define \x\ x //2 by (/ii x /i2)((si, s 2 )) = A*i( s i) x 1^2(82)- 
Following [2] we also define the interleaving of PAs: 

Definition 3. Let Pj = (Si,—>i,ISi,APi,Li) be two PAs with i = 1,2. The interleave 
composition V\ \ \ V2 is defined by: 

= (Si x S 2 ,->,ISi x 152, ^Pi x AP 2 ,L) 

where L((si,S2)) = £i(si) X £ 2 (s 2 ) and (si,s 2 ) — > fj, iff either si — > fii and /i = /ijX X> S2 , 
or s 2 ->■ /i2 and /i = £> Sl x /j, 2 . 

3.1. PCTL* and its sublogics. We introduce the syntax of PCTL [II] and PCTL* @] 
which are probabilistic extensions of CTL and CTL* respectively. PCTL* over the set AP 
of atomic propositions are formed according to the following grammar: 

V? ::= a I 9?i A 992 I -.99 I P M g(^) 

-0 ::= 99 I tpi A ?/>2 I I X -0 I -01 U "02 

where a G AP, 00 G {<,>,<,>}, g G [0,1]. We refer to ip and ip as (PCTL*) state and 
path formulae, respectively. 

The satisfaction relation s \= (p for state formulae is defined in a standard manner for 
boolean formulae. For probabilistic operator, it is defined by s \= P^^) iff \/o-.Prob ajS {{uj G 
Path(s) I uj \= -0}) to Q- The satisfaction relation ui \= ip for path formulae is defined exactly 



By standard measure theory this algebra is a a-algebra and all its elements are the measurable sets of 
paths. 
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the same as for LTL formulae, for example a; |= X ^ iff w 1 1 \= ip, and cj \= U tp2 iff there 
exists j > such that co\j \= ip2 and oj\k \= tpi for all < k < j. 

Sublogics. The depth of path formula if) of PCTL* free of U operator, denoted by Depth(ip), 
is defined by the maximum number of embedded X operators appearing in tjj, that is, 

• Depth((p) = 0, 

• Depth(ipi A 1P2) = max{Depth(ipi) , Depth(ip2)}, 

• Depth(^Tp) = Depth(ip) and 

• DepthQtip) = 1 + Depth{ip). 

Then, we let PCTL* - be the sublogic of PCTL* without the until (ipi U 1P2) operator. 
Moreover, PCTL* - is a sublogic of PCTL* - where for each tp we have Depth (if)) < i. 
The sublogic PCTL is obtained by restricting the path formulae to: 

ip ::= X (p I (pi U Lf2 I <p\ U- n if2 

Note the bounded until formula does not appear in PCTL* as it can be encoded by nested 
next operator. PCTL - is defined in a similar way as for PCTL* - . Moreover we let PCTL - 
be the sublogic of PCTL - where only bounded until operator p\\J-^ if2 with j < i is 
allowed. 

Logical equivalence. For a logic C, we say that s and r are /^-equivalent, denoted by s ~£ r, 
if they satisfy the same set of formulae of C, that is s \= tp iff r |= cp for all formulae ip in C. 
The logic C can be PCTL* or one of its sublogics. 

3.2. Strong Probabilistic Bisimulation. In this section we introduce the definition of 
strong probabilistic bisimulation |22j . Let {s — > be a collection of transitions of V, 

and let {pi}i<=i be a collection of probabilities with ^2 ie jPi = 1. Then (s, X^e/W**) ^ s 
called a combined transition and is denoted by s — >p [i where [i = ^ie/ Pif^i- 

Definition 4. An equivalence relation 7Z C 5 x S is a strong probabilistic bisimulation 
iff s 7£ r implies that L(s) = L(r) and for each s —¥ /i, there exists a combined transition 
r — )>p /u' such that \x 1Z fi' . We write s ~p r whenever there is a strong probabilistic 
bisimulation 1Z such that s 1Z r. 

It was shown in |22] that ~p is preserved by ||, that is, s ~p r implies s\\t ~p r\\t 
for any t. Also strong probabilistic bisimulation is sound for PCTL which means that if 
s ~p r then for any state formula cp of PCTL, s \= cp iS r \= (p. But the other way around 
is not true, i.e. strong probabilistic bisimulation is not complete for PCTL, as illustrated 
by the following example. 

Example 1. Consider again the two PAs in Fig. [1] and assume that L(s) = L(r) and 
L(s\) 7^ L{s2) 7^ L(ss). In addition, s±, S2, and S3 only have one transition to themselves 
with probability 1. The only difference between the left and right automata is that the 
right automaton has an extra step. It is not hard to see that s ~pctl* r - By Definition 
U s 00-p r since the middle transition of r cannot be simulated by s even with combined 
transition. So we conclude that strong probabilistic bisimulation is not complete for PCTL* 
as well as for PCTL. 

It should be noted that PCTL* distinguishes more states in a PA than PCTL. Refer to 
the following example. 
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Example 2. Suppose s and r are given by Fig. Q] where each of s±, 82, and S3 is extended 
with a transition such that s\ —> [i\ with /ii(si) = 0.6 and (11(34) = 0.4, S2 — > ^2 with 
^2(^4) = 1, and S3 — > /i3 with ^3(53) = 0.5 and ^3(54) = 0.5. Here we assume that 
every state satisfies different atomic propositions except that L(s) = L(r). Then it is 
not hard to see a ~pctl r while s "°pctl* r - Consider the PCTL* formula tp = 
P< . 38 (X(L(si) V L(s 3 )) A XX(L(si) V L(s 3 ))): it holds s \= tp but r tf= <P- Note that tp is 
not a well-formed PCTL formula. Indeed, states s and r are PCTL-equivalent. 

We have the following theorem: 

Theorem 1. (1) ~ PC tl, ~pctl*, ~pctl-> ~pctl~> ~pctl*-> ~pctl*-> and ~p are 
equivalence relations for any % > 1. 

(2) ~p C ~pcTL* Q ~PCTL- 

(3) ~pcTL*~ — ~PCTL"- 

(4) ~PCTL*~ = ~PCTLj"- 

(5) ~ PCTL *- C ~ PCTL - for any % > 1. 

(6) ~PCTL Q ~PCTL~ — ~PCTL^ 1 — ~PCTLr aU * — ®- 

(7) ~PCTL* Q ~PCTL*- ^ ~PCTLt- - ~PCTL*- for a11 * - °' 

Proof. We take ~pctl as an example and the others can be proved in a similar way. The 
reflexivity is trivial. If s ~pctl r , then we also have r ~pctl s since s and r satisfy the 
same set of formulae, we prove the symmetry of ~pctl- Now we prove the transitivity, that 
is, for any a, r, t if we have s ~pctl t and r ~pctl t, then s ~pctl t. It is also easy, 
since a and r satisfy the same set of formulae, and r and t satisfy the same set of formulae 
by a ~pctl t and r ~pctl t, as result a \= tp implies t \= tp and vice versa for any tp, so 
s ~pctl t. We conclude that ~pctl is an equivalence relation. 

The proof of ~p C ~pctl can be found in [22] while the proof of ~p C ~pctl* can be 
proved in a similar way. ~pctl* ^= ~pctl is trivial since PCTL is a subset of PCTL*. 

The proofs of Clause 3 and 5 are obvious since ~pcxl~ i s a subset of ~pctl*~ w hile 
~ PCTL - is a subset of ~pctl*~ ■ 

We now prove that ~pctl*~ = ~pctl~' ^ * s sufficient to prove that PCTL^ and 
PCTL^~ have the same expressiveness. ~pctl*~ — ~pctl~ * s eas y since PCTL^ is a 
subset of PCTLJ". We now show how formulae of PCTL* - can be encoded by formulae of 
PCTL^. It is not hard to see that the syntax of path formulae of PCTL^~ can be rewritten 
as: 

ip ::= tp I X(p I ->ip I ip± A ip2 
where we replace Xip with Xtp since PCTL^ - only allows path formulae whose depth is less 
or equal than 1. Since -1X99 = X-np, the syntax can refined further by deleting -up, that is, 

tp ::= tp I X<p I ipi A ip2 

Then the only left cases we need to consider are IVag (<£>), IPm^CX tp\ AX^), and Pm<j(X (pi A 

<&)> 

(1) a h V> q (<f) iSs\=tp, 

(2) a h P> g (X <piAX tp 2 ) iff a \= P> q (X(<pi A tp 2 )), 

(3) ^P> 5 (X^A^) iff a \=tp 2 A¥> q (Xtp 1 ). 

Here we assume that < q < 1, other cases are similar and are omitted. 

The proofs of Clauses 6 and 7 are straightforward. □ 
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4. A Novel Strong Bisimulation 

This section presents our main contribution of the paper: we introduce a novel notion of 
strong bisimulation and strong branching bisimulation. We shall show that they agree with 
PCTL and PCTL* equivalences, respectively. As the preparation step we introduce the 
strong 1-depth bisimulation. 



4.1. Strong 1-depth Bisimulation. 

Definition 5. A pre-order 1Z C S x S is a strong 1-depth bisimulation if s 1Z r implies that 
L(s) = L{r) and for any 1Z downward closed set C 

(1) if s — > /i with /i(C) > 0, there exists r — > // such that //(C) < /*(C), 

(2) if r — > fi with /J-(C) > 0, there exists s — >• fj,' such that //(C) < /t(C). 

We write s ~i r whenever there is a strong 1-depth bisimulation 1Z such that s 1Z r. 

The - though very simple - definition requires only one step matching of the distribu- 
tions out of s and r. The essential difference to the standard definition is: the quantification 
of the downward closed set comes before the transition s — > [i. This is indeed the key of the 
new definition of bisimulations. The following theorem shows that ~i agrees with ~pqxl~ 
and 

^PCTL* - ^hich. is also an equivalence relation: 
Lemma 1. ~ PCTL - = ~i = ~ PCTL *-- 

Proof. The proof of the first statement is trivial and is omitted here. 

The proof of the second statement is deferred to the proof of Theorem [3] and Theorem SJ 

□ 

Note that in Definition [5] we consider all the 1Z downward closed sets since it is not 
enough to only consider the 1Z downward closed sets in {sn \ s £ S}, refer to the following 
counterexample. 

Counterexample 1. Suppose that there are four absorbing states si,S2,S3, and S4 which 
are assigned with different atomic propositions. Suppose we have two processes s and r such 
that L(s) = L(r), and s — > m, s — > /i2, r — > u\, r — > v% where /ii(si) = 0.5, m(s2) = 0.5, 
^2(53) = 0.5, H2(si) = 0.5, ui(si) = 0.5, ^1(53) = 0.5, z^ 2 (s 2 ) = 0.5, ^2(^4) = 0.5. If we only 
consider the 1Z downward closed sets in {s^ | s S S} where S = {s, r, si, S2, S3, S4}, then we 
will conclude that s ~i r, but r \= ip while s \/= (p where 99 = P>o.s(X(L(si) V L(s 2 )))- 

It turns out that ~i is preserved by 1 1, implying that ~p CTL - and ~pctl* _ are preserved 
by 1 1 as well. 

Theorem 2. s ~i r implies that s\\t ~i r\\t for any t. 

Proof. We need to prove that for each ~i-closed set C, if s \ 1 1 — > fi such that fi(C) > 0, there 
exists r\\t — > // such that //(C) > /i(C) and vice versa. This can be prove by structural 
induction on s \ \ t and r \ \ t. By the definition of || operator, if s \ \ t — > /i, then either s — > /t s 
with fj, = (j, s || T> t , or t — > /it with fj, = T> s \ \ fit- We only consider the case when /j, = fi s \\V t 
since the other one is similar. We have known that s ~i r, so for each C if s — > fi s with 
Hs{C) > 0, then there exists r — > fj, r such that /t r (C) > n s (C). By induction, if s' ~i r' 
for s', r' € C, then s' \ \t ~i r' \ \t. So for each C and s \ \ t — > fj, with /i(C) > 0, there exists 
r || t // such that //(C) > fi(C). □ 
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Figure 2: ~ b is not compositional when i > 1 



Remark 1. We note that for Kripke structure (PA with only Dirac distributions) ~i agrees 
with the usual strong bisimulation by Milner [18] . 

4.2. Strong Branching Bisimulation. Now we extend the relation ~i to strong i-step 
bisimulations. Then, the intersection of all of these relations gives us the new notion of 
strong branching bisimulation, which we show to be the same as ~pctl- Recall Theorem [Tj 
states that ~pctl is strictly coarser than ~pctl*> which we shall consider in the next 
section. 

Following the way in [25] we define Prob atS (C,C ,n,u) which denotes the probability 
from s to states in C via states in C possibly in at most n steps under scheduler a, where 
u is used to keep track of the path and only deterministic schedulers are considered in the 
following. Formally, Prob ajS (C, C , n, to) equals 1 if s £ C", and else ifn>0A(sGC \ C"), 
then 

Prob a , s (C,C',n,uo) = ^ //(r) • Prob a>r (C, C , n - 1, ur). (4.1) 

r£supp(fi') 

where a(u>)(s, //) = 1, otherwise equals 0. 

Strong i-depth branching bisimulation is a straightforward extension of strong 1-depth 
bisimulation, where instead of considering only one immediate step, we consider up to i 
steps. We let ~^ = ~i m the following. 

Definition 6. A pre-order TZ C S x S is a strong i-depth branching bisimulation if i > 1 
and s TZ r implies that s r and for any TZ downward closed sets C, C, 

(1) if Prob a ^ s (C, C , i,s) > for a scheduler a, then there exists a scheduler a' such that 
Prob ali r(C,C',i,r) < Prob a>s (C,C ,i, s), 

(2) if Prob atr (C, C , i,r) > for a scheduler a, then there exists a scheduler a' such that 
Prob a ,J(C,C',i,s) < Prob a;r (C,C',i,r). 

We write s ~^ r whenever there is a strong i-depth branching bisimulation TZ such that 
s TZ r. The strong branching bisimulation ~ b is defined as ~ b = f\>i ~ b . 
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The following lemma shows that ~ b is an equivalence relation, and moreover, ~ b de- 
creases until a fixed point is reached. 

Lemma 2. (1) ~ b and ~ b are equivalence relations for any i > 1. 

(2) ~ b c ~ b provided that 1 < i < j. 

(3) There exists i > 1 such that ~ b = ~ b for any j,k >i. 

Proof. We only show the proof of transitivity of ~ b . Suppose that s ~ b t and t ~ b r, we 
need to prove that s ~ b r. By Definition [6l we know there exists strong i-depth branching 
bisimulations TZi and IZ2 such that s TZ\ t and t IZ2 r. Let 1Z = TZi o 7I2 = {(si,^) | 
3s2-(si TZi S2 A S2 7Z-2 r)}, it is enough to show that 7Z is a strong i-depth bisimulation. 
Note 7£i U 7^2 ^ 7£, since for each si TZi S2 we also have S2 7^-2 ^2 due to reflexivity, 
thus s\ 7Z S2, similarly we can show that 7^2 ^ 7£. Therefore for any 1Z downward closed 
sets C and C, they are also 7Z\ and 7^2 downward closed. Therefore if there exists a such 
that Prob atS (C, C , i) > 0, then there exists a' such that Prob a / t t{C, C , i) < Pro6 cr>s (C, C , i). 
Since we also have t ~ b r, thus there exists a" such that Prob a » >r (C, C, i) < Prob a ^ t {C, C , i) < 
Prob atS (C, C',i). This completes the proof of transitivity. 

The proof of Clause 2 is straightforward from Definition [HJ since s ~ b r implies 
s T when j > 1. 

It is straightforward from the Definition [6] that ~ b is getting more discriminating as i 
increases. In a PA only with finite states the maximum number of equivalence classes is 
equal to the number of states, as result we can guarantee that ~ b = ~ b where n is the total 
number of states. □ 

Let 1Z be an equivalence over S. The set C C S is said to be 7£-closed iff s £ C and 
s 7Z r implies r £ C. C-r is used to denote the least 7£-closed set which contains C . 

Definition 7. Two paths uj\ = sqS\ . . . and 0J2 = r o r i ■ ■ ■ are strong i-depth branching 
bisimilar, written as uj\ ~ b LJ2, iff 0J\\j\ ~ b [j] for all < j < i. 

The 7£-closed paths can be redefined based on Definition The set of paths is 
~ b -closed if for any uj\ £ Q, and ui2 such that u)\ ~ b UJ2, it holds that W2 E !!. Let 
B^h = {SI C B I Q. is ~ b -closed}. By standard measure theory Z3^b is measurable. The ~j 

z i 

for paths can be defined similarly and is omitted here. 

Lemma 3. s ~ b r implies that for each scheduler a\ and each Q £ B^b such that 
Prob a ^ s (CQ) > where = U C k C for two ~ b -closed sets C, C" with j < i, there exists 

0<k<j 

o~2 such that Prob a2jr (Cn) > Prob ai)S (Cfi) and vice versa. 

Proof. Note that by (|4.ip for any SI £ i3^b , if there exists j < i and ~ b -closed sets C, C 

i 

such that Q = U C k C, then Prob^ s (C,C ,j,s) = Prob a ^(Co). The following proof is 

0<k<j 

straightforward from Definitional □ 
Lemma 4. s ~pctl t iff s ~ b r for any n > 1, that is, ~pctl= H ~ b . 

n>l 

Proof. The proof is based on the fact that <p\ U 932 = U-°° 992 • D 
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It is not hard to show that ~ b characterizes PCTL^ . Moreover, we show that ~ b agrees 
with PCTL equivalence. 

Theorem 3. ~pcxl~ = ^ ^ or an y * — ^' an( ^ moreover ~pctl = ~ b - 

Proof. In the following, we will use Sat(p) = {s S 5 | s \= <p} to denote the set of states 
which satisfy p. Similarly, Sat(ip) = {oj € Path(so) \ uj \= ip} is the set of paths which 
satisfy ip. 

In order to prove that s ~pcpl- r implies s ~ b r for any s and r, we need to show that 
for any ~ pCTL - -closed sets C, C, if there exists a scheduler a such that Prob as (C, C ,j, s) > 
with j < i, then there exists a scheduler a' such that Prob G i r {C, C ,j, r) > Prob a ^ s (C, C ,j, s) 
and vice versa provided that s ~ PCTL - r - Suppose there are n different equivalence 
classes in a finite PA. Let Pd,Cj be a state formula such that Sat(pd,Cj) 5 Cj and 
Sat(ipc u Cj) H Cj = 0, here 1 < i / j < n and Ci,Cj G 5/ ~pcpl- are two different 
equivalence classes. Formula like <Pd,Cj always exists, otherwise there will not exist a for- 
mula which is fulfilled by states in d, but not fulfilled by states in Cj, that is, states in 
C{ and Cj satisfy the same set of formulae, this is against the assumption that Cj and Cj 
are two different equivalence classes. Let (pc t = A ipa d > it is not hard to see that 

Sat(ipd) = Ci. For a ~ pcTL - -closed set C, it holds 

Vc = V <pa, 

then Sat(ipc) = C. Now suppose Prob ajS (C,C ,j,s) = q > with j < i, then we know 
s \= -^P <q ip where 

ip = <pc U-- 5 (pa- 

By assumption r |= -^P <g ip, so there exists a scheduler a' such that Prob a i ^ r {C, C',j, r) > q, 
that is, Prob a r r (C,C',j,r) — Prob a ^ s {C,C' , j, s). The other case is similar and is omitted 
here. 

The proof of ~ b C ~ PCTL - is by structural induction on the syntax of state formula 

<p of PCTL^~ and path formula tp of PCTL t ~, that is, we need to prove the following two 
results simultaneously. 

(1) s ~ b r implies that s \= ip iff r |= (p for any state formula <p of PCTL^~. 

(2) ui\ ~ b 002 implies that u>\ \= ip iff 0J2 \= ip for any path formula ip of PCTL". 

We only consider <p = P> g (ip) here, s \= p iff Vcr.Pro&o- jS ({u; | to \= tp}) > q. The set fi 
of paths satisfying ip E 5eg 4 ~, O = | cj |= ip}, is ~ b -closed by the induction hypothesis. 
If ip = Xtp', the proof is obvious since ~ b implies ~ b . Suppose ?/> = (pi\J- 3 <p2 with 
j < i, we need to show that < i and there exists two ~ b -closed sets C,C such that 
O = U C k C, this is straightforward by the semantics of L)-- 7 . By Lemma [3] it follows 

0<fc<j 

that for each scheduler cri and each £ b such that O = U C fe C with j < i, there 

* 0<k<j 

exists o"2 such that Prob a2tr (Cn) > Proba- ltS (Cn) and vice versa. As result r \= (p. 

To prove ~pcTL=~ b we show first a lemma. We let ~ b = n ~ b in the following. 

n>l 

The proof of ~pcTL=~ b is straightforward by using Lemma [2] and Lemma [H 
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Intuitively, since ~ PCTL - = ~i* decreases with i, for any PA ~^ will eventually converge 
to PCTL equivalence. 

Recall ~^ is compositional by Theorem [2j which unfortunately is not the case for 
with i > 1. This is illustrated by the following example: 

Counterexample 2. s ^ r does not imply s\\t ~^ r || t for any t generally if i > 1. 

We have shown in Example [T] that s ~pctl r - If we compose s and r with i where 
t only has a transition to [i such that /u(ii) = 0.4 and \i{t-i) = 0.6, then it turns out that 
s || t "°pctl r || t. Since there exists ip = P<o.34'0 with 

V> = ((L(s\\t) V L( Sl || t) V (L(s 3 ||t))) U^ 2 (L( Sl || fc) V L(s 3 || *i))) 

such that s || t |= ip but r || t ft= ip, as there exists a scheduler <r such that the probability 
of paths satisfying ip in Prob a ^ r equals 0.36. Fig. [2] shows the execution of r guided by the 
scheduler a, and we assume all the states in Fig. [2] have different atomic propositions except 
that L(s || t) = L(r \ \ t). It is similar for ~pctl*- 

Note that ip is also a well- formed state formula of PCTL2 , so ~ PCTL - as well as ~]p 
are not compositional if i > 2. 

4.3. Strong Bisimulation. In this section we introduce a new notion of strong bisimula- 
tion and show that it characterizes ~pctl*- Given a pre-order TZ, a TZ downward closed 
cone Cq and a measure Prob, the value of Prob(Co) can be computed by summing up the 
values of all Prob{C w ) with uj € Vt. We let C TZ* be a set of TZ downward closed paths, 
then Cq is the corresponding set of TZ downward closed cones, that is, Cq = Uq g q Cfi- De- 
fine Z(f2) = Max{/(r2) I € 0} as the maximum length of f2 in $7. To compute Prob{C^), 
we cannot sum up the value of each Prob(Cn) such that S! £ !] as before since we may 
have a path u such that uj € f^i and w S where Oi, f^2 € 0, so we have to remove these 
duplicate paths and make sure each path is considered once and only once as follows where 
we abuse the notation and write uj £ Cl iff 30.(17 6 f2 A uj € fi): 

Pr &(C^)= ^ ProHOJ (4.2) 

wef2Apa/en.u/<w 

Note Equation 14.21 can be extended to compute the probability of any set of cones in a given 
measure. 

The definition of strong z-depth bisimulation is as follows: 

Definition 8. A pre-order TZ C S x S is a strong i-depth bisimulation if i > 1 and s TZ r 
implies that s r and for any f2 C 7£* with Z(J2) = i 

(1) if Prob atS (C^) > for a scheduler a, there exists a' such that 

Prob a ,,r{Cn) < Probed), 

(2) if Prob at r(C^) > for a scheduler a, there exists a' such that 

Prob a ,J(C^) < Prob a , r (Cn)- 
We write s ~« r whenever there is a i-depth strong bisimulation TZ such that s TZ r. The 
strong bisimulation ~ is defined as ~ = flj>i ~j. 

Similar to the relation ~j forms a chain of equivalence relations where the strictness 
of ~j increases as i increases, and ~j will converge finally in a PA. 

Lemma 5. (1) ~j is an equivalence relation for any i > 1. 
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(2) ~j C ~j provided that 1 < i < j. 

(3) There exists i > 1 such that ~j = f° r an Y j, k > i. 

Proof. The proof is similar with the proof of Lemma [2] and is omitted here. 

Lemma 6. s ~, r implies that for each scheduler o~\ and Cq such that C such that 
Z(fi) < i, there exists o"2 such that Prob^^C^) > Prob ai ^{C^) and vice versa. 

Proof. The proof is straightforward from Definition [8j □ 



Let ~= fl ~ n , we have a lemma as follows: 

n>l 



Lemma 7. s ~pctl* r iff s ~ n r for any n > 1, that is, ~pctl* = H ~ n . 

n>l 

Proof. The proof is similar with the proof of Lemma [H □ 

Below we show that ~, characterizes ~pcxl*~ ^ or a h i > 1, and ~ agrees with PCTL* 
equivalence: 

Theorem 4. ~pqxl*~ = ~» ^ or an y * — 1> an< ^ moreover, ~pctl* = ~- 

Proof. In order to prove that s ~pctl*~ r i m P nes s ~i r f° r an Y s an d r, we need to 
show that for any f2 C ~ pcTL „_* with < i, if there exists a scheduler er such that 

Prob aiS {C^) > 0, then there exists a scheduler <r' such that Prob a > r {C^) > Prob a ^{C^) 
and vice versa provided that s ~pctl*~ r ~ Following the way in the proof of Theorem [3j 
we can construct a formula ipc such that Sat(<pc) — C where C is a ~ pcTL *- -closed set. 
Suppose f2 = CqC\ . . .Cj with j < i, then 

Vto = ^C A X (m A ... A X{ipCj-i A X <£ Cj . )) 
can be used to characterize fi, that is, Sat{ip$i) = Cn- Let ^ = V_ ipQ, then Sat(ip) = Cq. 

As result s (= —iP <q ip where q = Prob atS {C^). By assumption r \= -^P <q ip, so there exists 
a scheduler a' such that Prob a i , r (Cft) > 9) that is, Prob a > , r (C^) > Prob a ^{C^). The other 
case is similar and is omitted here. 

The proof of ~j C ~ pcTL *_ is by structural induction on the syntax of state formula 

(p of PCTL*~ and path formula -0 of PCTL*~, that is, we need to prove the following two 
results simultaneously. 

(1) s ~j r implies that s \= ip iff r |= ip for any state formula (p of PCTL* - . 

(2) wi ~j W2 implies that wi |= ip iff W2 |= ip f° r an Y P a th formula ip of PCTL*~. 

We only consider ip = F> q (ip) here, s (= <^ iff Vcr.Pro6 - iS ({a; | w (= > g. The set of 
paths satisfying t/; € Seq*~ , = {w | w |= V>}, is ~j-closed by the induction hypothesis, and 
also < i since the depth of ^ is at most i. By Lemma[6]it follows that for each scheduler 
o"i and each Q C with Z(fi) < i, there exists o~i such that Prob a2 ^ r (C^) > Prob ai ^ s (C^) 
and vice versa. As result r |= 93. 

The proof is straightforward by using Lemma [5] and Lemma [7J □ 
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Figure 3: Relationship of different equivalences in strong scenario. 
Recall by Lemma there exists i > such that ~pctl* = ~«- 

For the same reason as strong z-depth branching bisimulation, ~j is not preserved by 
|| when i > 1. 

Counterexample 3. s ~j r does not imply s\\t ~j r\\t for any t generally if i > 1. 
This can be shown by using the same arguments as in Counterexample [2j 



4.4. Taxonomy for Strong Bisimulations. Fig. [3] summaries the relationship among all 
these bisimulations and logical equivalences. The arrow — > denotes C and -/» denotes ^. We 
also abbreviate ~pctl as PCTL, and it is similar for other logical equivalences. Congruent 
relations with respect to || operator are shown in circles, and non-congruent in boxes. Segala 
has considered another strong bisimulation in [22], which can be defined by replacing the 
r — >p fx' with r — > // and thus is strictly stronger than ^p. It is also worth mentioning that 
all the bisimulations shown in Figj3] coincide with the strong bisimulation defined in [3] in 
the DTMC setting which can be seen as a special case of PA (i.e., deterministic probabilistic 
automata) . 

5. Weak Bisimulations 

As in [3] we use PCTL\x to denote the subset of PCTL without next operator X{p and 
bounded until ip\ U- n (f2- Similarly, PCTL* X is used to denote the subset of PCTL* with- 
out next operator X^. In this section we shall introduce weak bisimulations and study 
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the relation to ~pctL\ X an< ^ ~pctl^ x i respectively. Before this we should point out that 
~pctl^ x implies ~pctL\ X but the other direction does not hold. Refer to the following 
example. 

Example 3. Suppose s and r are given by Fig. [T] where each of si and S3 is attached 
with one transition respectively, that is, s\ — > /xi such that ^{s^} = 0.4 and /xi(ss) = 0.6, 
■?3 such that //3(s4) = 0.4 and ^3(55) = 0.6. In addition, S2, s 4 and S5 only have 

a transition with probability 1 to themselves, and all these states are assumed to have 
different atomic propositions. Then s ~pctL\ X t but s ^pctl^ x r , since we have a 
path formula V = ((L(s) V L(s{)) U L(s 5 )) V ({L(s) V L(s 3 )) U L(s 4 )) such that s \= F< Q . 3A ip 
but r \/= lP<o.34V ; ! since there exists a scheduler a where the probability of path formulae 
satisfying ip in Prob a ^ r is equal to Prob a ^ r {ssis^) + Prob a ^{ss 3 Si) = 0.36. Note ip is not a 
well-formed path formula of PCTL^x- 

5.1. Branching Probabilistic Bisimulation by Segala. Before introducing our weak 
bisimulations, we give the classical definition of branching probabilistic bisimulation pro- 
posed in [22]. Given an equivalence relation TZ, s can evolve into fi by a branching transition, 
written as s =^ fi, iff i) \x = T> s , or ii) s — >■ // and 

rG(s«pp(/^ / )n[s])A'r=^' R -/x r rEsupp(/j,')\[s] 

where [s] denotes the equivalence class containing s. Stated differently, s =^ \i means 
that s can evolve into [i only via states in [s]. Accordingly, branching combined transition 
s =>p^ [i can be defined based on the branching transition, i.e. s =>p \i iff there exists a 
collection of branching transitions {s =^ ^}j g /, and a collection of probabilities {pi}i^i 

witn Yli&lPi = 1 sucn tliat A 4 = Yli&lPi^i- 

We give the definition branching probabilistic bisimulation as follows: 

Definition 9. An equivalence relation TZ C 5 x S 1 is a branching probabilistic bisimulation 
iff s 7£ r implies that L(s) = L(r) and for each s — >• /x, there exists r =>p // such that 

We write s ~p r whenever there is a branching probabilistic bisimulation TZ such that 
sTZr. 

The following properties concerning branching probabilistic bisimulation are taken 
from [22]: 

Lemma 8 ([22]). (1) ~ P C ~ PC tl* x C ~pctl xx - 
(2) ~p is preserved by ||. 

5.2. A Novel Weak Branching Bisimulation. Similar to the definition of bounded 
reachability Prob ajS (C,C ,n,uj), we define the function Prob ajS (C,C ,00) which denotes the 
probability from s to states in C possibly via states in C. Again ui is used to keep track 
of the path which has been visited. Formally, Prob a>s {C,C' ,00) is equal to 1 if s G C, 
Prob ajS (C,C' is equal to if s ^ C, otherwise when a(co)(s, //) = 1, 

Prob a , s {C,C',uj) = fJ,'(r)- Prober (C,C',ur). (5.1) 

r£supp(fi') 
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The definition of weak branching bisimulation is as follows: 

Definition 10. A pre-order 1Z C S x S is a weak branching bisimulation if s 1Z r implies 
that L(s) = L(r) and for any 7Z downward closed sets C, C 

(1) if Prob a;S (C, C , s) > for a scheduler a, there exists a' such that 
Prob a , jr \c,C',r) < Prob^iCC'^s), 

(2) if Prob atr (C, C , r) > for a scheduler a, there exists a' such that 
Pro& CT , iS '(C7,C7',s) < Prob^ r (C,C ». 

We write s ~ b r whenever there is a weak branching bisimulation 7Z such that s 1Z r. 

The following theorem shows that ~ b is an equivalence relation. Also different from 
the strong cases where we use a series of equivalence relations to either characterize or 
approximate ~pctl and ~pctl*j m the weak scenario we show that ~ b itself is enough 
to characterize ~pctl xx - Intuitively because in ~pctl xx only unbounded until operator is 
allowed in path formula which means we abstract from the number of steps to reach certain 
states. 

Theorem 5. (1) ~ b is an equivalence relation. 
(2) pa = ~pcTL\ X - 

Proof. (1) The reflexibility of ~ b is trivial. The symmetry of « b is straightforward 
from Definition fTTJl Suppose that s ~ b r and r ~ b t, then for any ~ b - 
closed sets C, C, if Prob a!S (C, C, s) > for a scheduler a, there exists a' such that 
Prober tr (C,C' ,r) > Prob a:S (C,C' , s). Since we also have r w b t, so there exists 
a" such that Prob c » }t {C,C' ,t) > Prob a , , r (C,C" » > Prob^C, C , s). Similarly if 
Prob a j(C,C ,t) > for a scheduler a, there exists a' such that Prob a i >S (C,C' , s) > 
Prob a j(C,C' ,t). This proves the transitivity of ~ b . 
(2) In order to prove that s ~pctL\ X r implies s « b r for any s and r, we need to 
show that for any ~pcTL\ X -closed se ^ s ^ C ■> ^ there exists a scheduler a such that 
Prob a ^(C,C , s) > 0, then there exists a scheduler cr' such that Prob a ' ir (C,C ,r) > 
Probcr tS (C,C' , s) and vice versa provided that s ~pcTLy X r. Following the way 
in the proof of Theorem O we can construct a formula (/9c such that Sat(<pc) = C 
where C is a ~pctL\ X -closed set. Let ip = fc^fC'i then it is not hard to see 
that s \= -^F <q ip where q = Prob a>s (C,C' , s). By assumption r |= -iP <q ip, so there 
exists a scheduler cr' such that Prob a ' >r (C,C' , r) > q, that is, Prob a i >r (C,C',r) > 
Prob aiS (C, C ,s). The other case is similar and is omitted here. 

The proof of ~ b C ~pcTL\ X is by structural induction on the syntax of state 
formula (p of PCTLyx and path formula ip of PCTL\x> that is, we need to prove 
the following two results simultaneously. 

(a) s ~ b r implies that s \= ip iff r |= ip for any state formula ip of PCTL\x- 

(b) uj\ ~ b UJ2 implies that uj\ \= ip iff ui2 \= ip f° r any path formula ip of PCTLyx- 
We only consider ip = ¥> q {ip) with ip = ip± U <pi here, s |= iff Va.Probp^^L} \ 
w \= ip}) > q- {<^> | oj \= ip}, Sat(ipi), and Sat(ip2) are « b -closed by the induction 
hypothesis, moreover Prob atS ({uj \ u \= ip}) = Prob atS (Sat(ipi), Sat(ip2), s) by Equa- 
tion ()5.1|) for any a. So for each a\ such that Prob ai ^ s (Sat(ipi), Sat {(^2)1 s) > 0, there 
exists (72 such that Prob a2tr (Sat(ipi), Sat(ip2),r) > Prob ai ^ s (Sat(ipi), Sat(ip2), s) and 
vice versa. As result r \= ip. 

□ 
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As in the strong scenario, ~ b suffers from the same problem as ~ b and ~j with i > 1, 
that is, it is not preserved by ||. 

Counterexample 4. s ~ b r does not always imply s\\t ~ b r\\t for any t. This can 
be shown in a similar way as Counterexample [2] since the result will still hold even if we 
replace the bounded until formula with unbounded until formula in Counterexample [2j 



5.3. Weak Bisimulation. In order to define weak bisimulation we consider stuttering 
paths. Let O be a finite TZ downward closed path, then 

'Ca i(n) = i 

U C(n[o]) fe o...(n[n-2l) fc "-2Q[n-i] l(Q) = n>2 ( 5 - 2 ) 

^VO<i<n.Vfei>0 

is the set of TZ downward closed paths which contains all stuttering paths, where fi[£] denotes 
the (i + l)-th element in Q such that < % < Accordingly, CU = U Co, st contains 

all the stuttering paths of each f2 E Q. Given a measure Prob, Prob{Q s t) can be computed 
by Equation (JO]). 

Now we are ready to give the definition of weak bisimulation as follows: 

Definition 11. A pre-order TZ C S x S is a weak bisimulation if s 7£ r implies that 
L(s) = L(r) and for any Q C 72* 

(1) if Proba^C^ ) > for a scheduler <r, there exists cr' such that 

(2) if Prob atr (CQ ) > for a scheduler cr, there exists a' such that 

We write s ~ r whenever there is a weak bisimulation TZ such that s TZ r. 

The following theorem shows that ~ is an equivalence relation. For the same reason as 

in Theorem [5l ~ is enough to characterize ~pctl* which gives us the following theorem. 

\ x 

Theorem 6. (1) ~ is an equivalence relation. 
(2) w = ~ PCTL . x . 

Proof. (1) The proof is similar with Clause 1 of Theorem [5] and is omitted here. 

(2) In order to prove that s ~pctl* r implies s ~ r for any s and r, we need to show 

that for any Cl C ~pctl^ x *> if there exists a scheduler a such that Prob a:S (C^ st ) > 0, 

then there exists a scheduler a' such that Prob a > tr (CQ ) > Prob a<s (CQ ) and vice 

versa provided that s ~pctl* r - Following the way in the proof of Theorem [3l we 

\ x 

can construct a formula ipc such that Sat(ipc) = C where C is a ~pctl* -closed 

\ x 

set. Let ifj n = <p Co U . . . ip Cn where Q = C Co ...c n , then ip^= V Vn- So s |= -^ <q ip 

where q = Prob atS (C^ ) and ip = ip^. By assumption r |= -iP <(? V ; 5 so there exists 
a scheduler a' such that Prob a i ,r(Cn ) > 9; that is, Prob a ' tr (C^ st ) > Prob a ^ s {C^ st ). 
The other case is similar and is omitted here. 

The proof of ~ C ~pctl* is by structural induction on the syntax of state 

\ x 

formula ip of PCTL* X and path formula tp of PCTL* X , that is, we need to prove 
the following two results simultaneously. 
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(a) s ~ r implies that s \= <p iff r \= p for any state formula ip of PCTL* X . 

(b) u)\ ~ C02 implies that us\ \= tp iff 0J2 (= V* f° r an y path formula t/> of PCTL* X . 
To make the proof clearer, we rewrite the syntax of PCTL* X as follows which is 

equivalent to the original definition. 

ip ::= cp I tpi V ip2 I I 01 U 02 
We only consider <p = P> q (tp) here. We need to prove that for each a for each 
ip, there exists f2 C cs 00 such that Pro6 CT)S (0) = Prober, s(Sat(tp)). The proof is by 
structural induction on tp as follows: 

(a) ip = <p' '. By induction Sat(p') is ^-closed. Let Cl = {Sat(p')}, then Prob a , s (£l) = 
Prob a>s {Sat{ip)). 

(b) ip = ipi Vip2- By induction there exists £1' and Vt" such that Prob a ^ s {Sat{ip\)) = 
Prob^g^Q, ) and Prob atS (Sat{tp2)) = Prob atS (C^„ ). It is not hard to see that 

Q, = O' U fi" will be enough. 

(c) V = "01 U "02- By induction there exists f2' and 17" such that Prob a ^ s {Sat{ip\)) = 
Prob^siC^ ) and Prob a>s (Sat(tp 2 )) = Prob a>s {C^,). Let = '{fl'n" | 0' G 
tt' A 0" G O"}, then Probity = Prob^ s {Sat{ip)). 

(d) tp = -iip'. s \= P> q (tp) iff s |= P<i_ (? ('i/' / )) so can be reduced to another 
formula without -1 operator. 

The following proof is routine and is omitted here. 

□ 

Not surprisingly ~ is not preserved by ||. 

Counterexample 5. s ~ r does not always imply s\\t ~ r || t for any t. This can be 
shown by using the same arguments as in Counterexample HI 

5.4. Taxonomy for Weak Bisimulations. As in the strong cases we summarize the 
relation of the equivalences in the weak scenario in Fig. H] where all the denotations have 
the same meaning as Fig. [3l Compared to Fig. [3l Fig. H] is much simpler because the step- 
indexed bisimulations are absent. As in strong cases, here we do not consider the standard 
definition of branching bisimulation which is a strict subset of ~p and can be defined by 
replacing =4>p with in Definition [9j Again not surprisingly all the relations shown in 
Fig. U] coincide with the weak bisimulation defined in [3] in DTMC setting. 

6. Simulations 

In this section we discuss the characterization of simulations w.r.t. the safe fragments of 
PCTL and PCTL*. First let us introduce the safe fragment of PCTL*, denoted by PCTL*, 
which is defined by the following syntax: 

ip ::= a \ \ p>\ A <p 2 \ Pi V P2 \ P>gC0) 

tp ::= p \ tpi A ip2 I "01 V "02 I X^> \ tpiUtp2 

where a G AP and q G [0, 1]. Accordingly the safe fragment of PCTL, denoted by PCTL S , 
is a sub logic of PCTL* where only the path formula is constrained to be the following form: 

ip ::= X (p I (pi U (p2 I <pi U- n p>2- 
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Figure 4: Relationship of different equivalences in weak scenario. 

We write s -<pctl* r iS r \= (p implies that s \= (p for any ip of PCTL*, and similarly 
for other sub-logics. 

Again we first introduce the strong probabilistic simulation introduced in [22] before 
doing so we need to define the weight function in the way as p3| . 

Definition 12. Let 1Z = S x S be a relation over S. A weight function for \x and v with 
respect to 1Z is a function A : S x S i->- [0, 1] such that: 

• A(s, r) > implies that s 1Z r, 

• v(r) = ^ sgS A(s,r) for any r G S. 

We write /i C-^ v iff there exists a weight function for [i and v with respect to 7£. 

Below follows the definition of strong probabilistic simulation. 

Definition 13. A relation 1Z C 5 x S is a strong probabilistic simulation iff s 7?. r implies 
that L(s) = L(r) and for each s — >• there exists a combined transition r — s-p // such that 
A 4 '- We write s -<p r whenever there is a strong probabilistic simulation 1Z such 
that s 7Z r. 

It was shown in [22] that is congruent, i.e. s -<p r implies that s\\t -<p r\\t 
for any i. But not surprisingly, it turns out that the strong probability simulation is too 
fine w.r.t -<;pcTL a an d ^pctl* which can be seen from Example [TJ Similarly we have the 
correspondent theorem of Theorem Q] in the simulation scenario where we only consider the 
safe fragment of the logics, thus the subscription s is often omitted for readability. 

Theorem 7. (1) ^ PC tl, ^pctl*, ^pctl-> ^pctl~> ^pctl*-' ^pctl*-> and are 

i i 

pre-orders for any i > 1. 
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Proof. For Clause (1) we only prove that -<pctl is a preorder since the others are similar. 
The reflexivity is trivial as s -<pctl s for any s. Suppose that s -<pctl t and t -<pctl r, 
then we need to prove that s ^pctl r in order to the transitivity. According to the 
definition of ^pctl, we need to prove that r \= ip implies s \= (p for any ip. Suppose that 
r |= tp for some p, then t \= ip because of t ^pctl r, moreover since s -<pctl t, hence 
s \= (p which completes the proof. 

The proof of Clause (2) can be found in [22]. Since we have shown in Theorem Q] 
that PCTLj - and PCTL^ - have the same expressiveness, thus the proof of Clause (4) is 
straightforward. The proofs of all the other clauses are trivial. □ 

6.1. Strong i-depth Branching Simulation. Following Section 2^2] we can define strong 
i-depth branching simulation which can be characterized by -^pcrrr- -^ e ^ s r iff 

L(s) = L(r), then 

Definition 14. A relation TZ C S x S is a strong i-depth branching simulation with 
i > 1 iff s TZ r implies that s -< b _i r and for any TZ downward closed sets C,C, when- 
ever Proba- tS (C, C',i) > for a scheduler a, there exists a' such that Prob a ' r (C,C ,i) < 
Prob a>a (C,C' ',£). We write s -<; b r whenever there is a strong i-depth branching simulation 
TZ such that s TZ r. The strong branching simulation -< b is defined as -< b = Hj>o 

Below we show the similar properties of strong i-depth branching simulations. 

Lemma 9. (1) -< h and -< b are pre-orders for any i > 0. 

(2) -<) C -< b provided that < i < j. 

(3) There exists i > such that -< b = for any j,k > i. 

Proof. (1) The reflexivity is trivial, we only prove the transitivity. Suppose that si -< b S2 
and S2 -< b S3, we need to prove that s± -<; b S3. By Definition 1141 there exists 
strong simulation TZ± and 7^-2 such that s\ 1Z\ S2 and S2 IZ2 S3. Let TZ = TZ\ o 7£.2 = 
{(si,S3) I 3s2-(si 7^-1 S2 A S2 72-2 S3)}, it is enough to prove that TZ is strong i- 
depth branching simulation. Due to the reflexivity, any TZ downward closed set C 
is also TZi and 7^2 downward closed. Therefore for any TZ downward closed sets 
C,C, if Probcr jS1 (C,C' ,i) > for a scheduler a, then there exists a' such that 
Prob a > )S2 (C, C" \i) < Prob CTtS1 (C,C ,i) according to Definition [T4l Similarly, there 
exists a" such that Prob a " )S3 (C,C ,i) < Prob a > )S2 (C,C ,i) < Prob a>sl (C,C and 
TZ is indeed a strong z-depth branching simulation. This completes the proof. 

(2) It is straightforward from Definition 1141 

(3) Similar with the proof of the third clause of Lemma EJ and is omitted here. 

□ 

Our strong i-depth branching simulation coincides with ^pctl - ^ or eac ^ h therefore 

i 

"^pctl is equivalent to -<! b as shown by the following theorem. 
Theorem 8. ^pqxl~ = ^ or an y * — 1> an d moreover ^pctl = 

i 

Proof. We first prove that ^ PCTL - implies -< b . Let TZ = {(s,r) | s -<p CTL - r }> h is enough 
to prove that TZ is a strong i-depth branching bisimulation. Suppose that s TZ r, we need 
to prove that for any TZ downward closed sets C, C and scheduler a of s, there exists a 1 of 
r such that Prob a i r (C,C ,i) < Prob a S (C, C" Note that Sat(ip) is a TZ downward closed 
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set for any ip. Since the states space is finite, for each TZ downward closed set C, there 
exists ifc such that Sat(cpc) = C. Assume that there exists TZ downward closed sets C,C 
and a such that Prob a > ir (C,C ,i) > Prob a)S (C,C ,i) for all schedulers a' of r. Then there 
exists q such that r |= F> q (ip) but s ft= F> q (tp) where ip = tpc U- 1 ipc, this contradicts with 
the assumption that s -^pctlt - r - Therefore TZ is a strong i-depth branching bisimulation. 

In order to prove that implies ^ PCTL -> we need to prove that whenever s r 
and r \= ip, we also have s \= ip. We prove by structural induction on (p, and only 
consider the case when ip = F> q (<pi \J- 1 ^2) since all the others are trivial. By induc- 
tion Sat(ipi) and Sat{p2) are -<\ downward closed, therefore if r \= P> g (<^i U- J P2), but 
s Y= F> q (<p\ U— ' (P2), then there exists a of s such that there does not exist a' such that 
Prob a >^ r (Sat(tpi), Sat((p2),i) < Prob atS (Sat(ip%), Sat(<p2),i) which contradicts with the as- 
sumption that s -<\ r. □ 

In Counterexample [2] we have shown the ~]? is not compositional for i > 1, using the 
same arguments we can show that is not compositional either for i > 1, thus we have 

Theorem 9. s -<\ r implies that s\\t -<\ r\\t for any t, while with i > 1 is not 
compositional in general. 



6.2. Strong f-depth Simulation. In this section we introduce strong i-depth simulation 
which can be characterized by ^p^pL* - wri ere we omit the proofs of the lemmas and 
theorems since they can be proved in a similar way as in Section 16.11 Below follows the 
definition of strong i-depth simulation where -<o — ^o- 

Definition 15. A relation TZ C S x S is a strong i-depth simulation with i > 1 iff s TZ r 
implies that s -<i—i r and for any £1 C TZ* with Z(fi) = i, if Prob a ^ s {C^) > for a scheduler 
a", there exists a' such that Prob a ' ,r(Cfj) < Fro6 CT)S (C^). 

We write s -<j r whenever there is a i-depth strong simulation 7£ such that s TZ r. 
The strong simulation -< is defined as ~< = flj>o -<j. 

Below we show the similar properties of strong i-depth simulations. 

Lemma 10. (1) -< and -<j are pre-orders for any i > 0. 

(2) -<j C provided that < i < j. 

(3) There exists i > such that = -<t for any j, k > i. 

Our strong i-depth simulation coincides with -<p CTL *- for each i, therefore -<pctl* is 
equivalent to -< as shown by the following theorem. 

Theorem 10. ^ PCTL *- = ~<i fo r any * > lj and moreover ^pctl* — ~<- 

Similarly, we can show that -<j is not compositional either for % > 1, thus we have 

Theorem 11. s -<i r implies that s\\t -<\ r\\t for any t, while -<i with i > 1 is not 
compositional in general. 
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6.3. Weak Simulations. Given the results for weak bisimulations from Section [5j the 
characterization of weak simulations is straightforward. Let us first introduce the definition 
of branching probabilistic simulation by Segala as follows: 

Definition 16. A relation TZ C S x S is a branching probabilistic simulation iff s TZ r 
implies that L(s) = L(r) and for each s — > fj,, there exists r =^p // such that fi TZ fjf. 

We write s r whenever there is a branching probabilistic bisimulation TZ such that 
s 1Z r. 

From |22j we know that is compositional, but it is too fine for ^pctL\ X as well as 

^pctl* i therefore along the line of weak bisimulations, we come out similar results for 

~ \ x 

weak simulations. Below follows the definition of weak branching simulation. 

Definition 17. A relation 7Z C 5 x S is a weak branching simulation iff s 1Z r implies 
that L(s) = L(r) and for any 1Z downward closed sets C,C, if Prob ajS (C,C , s) > for a 
scheduler a, there exists a' such that Prob a i ir (C,C',r) < Prob aiS (C,C , s). 

We write s ^ b r whenever there is a weak branching simulation 1Z such that s 1Z r. 

Due to Counterexample HI ^ b is not compositional, but it coincides with ^pctL\ X as 
shown by the following theorem. 

Theorem 12. ^ b is a pre-order, and ^ b = ~pctL\ X - 

The weak simulation equivalent to ~pctl* can also be obtained in a straightforward 

~ \ x 

way by adapting Definition [1TJ 

Definition 18. A relation 1Z C S x S is a weak simulation iff s 1Z r implies that L(s) = L{r) 
and for any Q C TZ* whenever Prob ajS (C^ ) > for a scheduler a, there exists a 1 such that 
Prob^^jKProb^C^J. 

We write s ^ r whenever there is a weak simulation TZ such that s TZ r. 

Again ^ is not compositional, but it coincides with ^pctl* j therefore we have the 
following theorem. 

Theorem 13. ^ is a pre-order, and ^ = ^pctl* • 



7. The Coarsest Congruent Bisimulations and Simulations 

Before we have shown that ~p is congruent but cannot be characterized by ~pctl com- 
pletely since it is too fine. On the other hand, there exists ~ b which can be characterized 
by ~pctl, but it is not congruent generally, this indicates that ~pctl is essentially not 
congruent. Therefore a natural question one may ask is that what is the largest subset of 
~PCTL which is congruent. The following theorem shows that ~p is such coarsest congruent 
relation in ~pctl- 

Theorem 14. ~p is coarsest congruent equivalence relation in ~pctl- 

Proof. We prove by contradiction. Suppose that there exists ~p C — C ~pctl such that ~ 
is congruent. Since ~p C — , there exists s and r such that s ~ r but s oo-p r. According 
to Definition HI there exists s — > // such that there does not exist r — >p v with fi ~p v. Let 
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Supp(p) = {s i, S2, • • • , s n } and p(sj) — with 1 < i < n. Without losing of generality we 
assume that for each two (combined) transitions of r: r — >j> v\ and r —>p there does not 
exist < wx, W2 < 1 such that wx + W2 = 1 and fi ~p (wx • vx + W2 • ^2) (every combined 
transition of r can be seen as a combined transition of two other combined transitions of r). 
Let vx( s i) = bi and ^(sj) = Q in the following, then there must exist 1 < j < n such that 
there does not exist < wx , W2 < 1 such that wx ■ b{ + W2 - Q = a» and 101 • 6j + ^2 ■ Cj = aj 
with it!i + W2 = 1, otherwise the combination of z^i and f2 will be able to simulate p. There 
are nine possible cases in total depending on the relation between a^aj and bi,Ci,bj,Cj. 
Most of the cases are impossible except when a, £ [bi, cj] and aj £ [cj, 6,]. For instance if 
aj > 6j,Cj, s will evolve into Sj with higher probability than r, thus s and r will not satisfy 
the same set of PCTL formulas i.e. s "°pctl r which contradicts with the assumption. 
Considering the following inequations: 

<k ■ Pi + aj ■ P2 > k ■ px + bj ■ p 2 , (7.1) 

Oj • pX + Oj • p 2 > Cj • pi + Cj • p 2 (7.2) 
which can be transformed into the following forms: 

(di - 6j) • px > (bj - aj) ■ p 2 , (7.3) 

(a; - Ci) ■ px > (cj - aj) ■ p 2 . (7.4) 

Note that (aj — fej), (a, — Cj), (6j — aj), and (cj — aj) cannot be at the same time, so there 
always exists < px , P2 < 1 such that aj • px + aj • p2 is either greater or smaller than both 
of bi - px + bj ■ p2 and Cj • pi + c,- • P2. Suppose the case such that if pi € ( ^ _? J • P2, ° J _!? J ■ P2) 
(it is not possible for j V = aj Cj , otherwise there exists < wx , w% < 1 such that 
wx-bi + W2 - Ci = aj and u>i • Oj + W2 ■ Cj = aj with wx + W2 = 1) , then Oi ■ px + aj ■ P2 is greater 
than bi- px + bj ■ P2 and Cj • px + Cj ■ P2 ■ Let t be a state such that it can only evolve into tx with 
probability px and t 2 with probability p2 where px + P2 = 1 and pi € ( ^ _^ • p2, • P2). 
Assume that all the states have distinct labels except for s and r, moreover let 

rp = {{L{s \\t) V L(s; || t) V [L{ S j \ \ t))) \J- 2 (L( Si \\ tx) V L( S j 1 1 1 2 ))), 

it is not hard to see that the maximum probability of the paths of s \ \ t satisfying tp is equal 
to aj • px + aj • P2 i.e. when s\\t first performs the transition s — > p of s and then performs 
the transition t — >• {pi :tx,p2 '■ ^2} of Since Oj • pi + aj • P2 is greater than 6j • pi + 6j • P2 
and Ci • pi + Cj ■ p2 , so the maximum probability of the paths of r\\t satisfying ip is less 
than s\\t, thus there exists <p = P< q ip such that r 1 1 1 |= ip but s 1 1 1 ^= <p which means that 
s\\t ^pctl r H*> as a result s\\t 9^ r\\t, so ~ is not congruent. This completes our 
proof. □ 

^For simplicity we assume that Sj(l < i < n) belong to different equivalence classes. 
4 We assume here that Cj > 6i and 6j > Cj 
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Theorem [TH can be extended to identify the coarsest congruent weak bisimulation in 
~PCTL\ X ' an d the coarsest congruent strong and weak simulations in -<pctl and ~pctL\ X 
respectively. 

Theorem 15. (1) ~p is coarsest congruent equivalence relation in ~pctL\ X » 

(2) -<p is coarsest congruent pre-order in -<pctl> 

(3) ^p is coarsest congruent pre-order in ^pcTL\ X - 

Proof. Similar with the proof of Theorem 1141 and is omitted here. □ 



8. Countable States 

For now we only consider finite PAs i.e. only contain finite states. In this section we will 
show that these results also apply for PAs with countable states. Assume S is a countable 
set of states S. We adopt the method used in [8] to deal with strong branching bisimulation 
since all the other cases are similar. First we recall some standard notations from topology 
theory. Given a metric space (S,d), a sequence {sj | i > 0} converges to s iff for any e > 0, 
there exists n such that d(s m ,s) < e for any m > n. A metric space is compact if every 
infinite sequence has a convergent subsequence. 

Below follows the definition of metric over distributions from [8]. 

Definition 19. Given two distributions /i, v E Dist(S), the metric d is defined by d(fj,, v) = 
Sup C€S \^C)-v{C)\. 

Since the metric is defined over distributions while in Definition [9] we did not consider 
distributions explicitly, thus we need to adapt the definition of Prob a)S (C,C , n) in the 

n C 

following way: s =^=> /x iff either i) \i = T> s , or ii) s — > v such that ^ v(r)-v r = 

. , ^ . , n—l.C 

vr£bupp(u).r =>• v r 
n C 

[i. It is obvious that for each o~,C,C, and n, there exists s ==> \i such that n(C) = 
Prob^C, C',n). 

Now we can define the compactness of probabilistic automata as [8] with slight differ- 
ence. 

i C 

Definition 20. Given a probabilistic automaton V, V is i-compact iff {/x | s ==4> //} is 
compact under metric d for each s € S and ~J? closed set C . 

As mentioned in [El [20], the convex closure does not change the compactness, thus we 

n C 

can extend ==> to allow combined transitions in a standard way without changing anything, 
but for simplicity we omit this. 

We introduce the definition of capacity as follows. 

Definition 21. Given a set of states 5 and a c-algebra B, a capacity on B is a function 
Cap : B -> (R + U {0}) such thafi 

(1) G>(0) = 0, 

(2) whenever d C C 2 with d,C 2 G B, then Cap{d) < Cap(d), 

(3) whenever there exists Ci C C 2 C . . . such that Uj>iCj = C, or C\ D C 2 5 ■ ■ ■ such 
that rii>iCj = C, then limj-j.oo Cap(d) = Cap(C). 



'R is the set of positive real numbers. 
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A capacity Cap is sub- additive iff Cap(C\ U C2) < Cap{C\) + Cap{p2) for any C\,C2 G /3. 

Different from [8], the value of Prob ajS (C,C , n) depends on both C and C". Let 
PreCap° n {C) = Sup a Prob aiS {C,C' ,n) and PostCap^' n (C) = Sup a Prob ajS (C,C » i.e. 
given a C" PreCapg n will return the maximum probability from s to C" in at most n 
steps via only states in C probably, similar for PostCap^ n . The following lemma shows 
that both PreCap^ n and PostCap^ n are sub-additive capacity. 

Lemma 11. PreCapg n and PostCap^ n are sub-additive capacity on B where B is a-algebra 
only containing rJ? closed sets. 

Proof. Refer to the proof of Lemma 5.2 in [8]. □ 

Now we can show that the following results are still valid as long as the given proba- 
bilistic automaton is compact even when it contains infinitely countable states. 

Theorem 16. Given a compact probabilistic automata, 

(1) ~n = ~PCTL~> 

(2) there exists n > such that = ~pctl- 
Proof. (1) The proof of ~> c 

~pctl- i s similar with the proof of Theorem El and is 
omitted here. We prove that ~pctl- — ~n m * ne sec L ue l following the proof of 
Theorem 6.10 in [HJ. Let 1Z = {(s,r) j s ~pctl~ r i> we nee d to prove that 1Z is 
a strong i-depth branching bisimulation. In order to do so, we need to prove that 
for any (s,r) £ 7Z, PreCapg n (C) = PreCapg n (C) for each 1Z closed sets C and 
C . Since both C and C may be countable union of equivalence classes while each 
equivalence class can only be characterized by countable many formulas, therefore we 
have C = U^n^dj) and C = Ug 1 (n^ 1 C( i ) where C\f =1 dj corresponds the 
i-th equivalence class in C, and Cjj corresponds the set of states determining by the 
j-th formula satisfied by i-th equivalence class, similar for n^ =1 C^ • and •. Similar 
as 0, let B k = n^Udj), A{ = Pi^U^Gj), and B' k = n^U^C^), 
A£ = rij =1 (uJ : =1 C J ' J ). It is easy to see that B k and B' k are increasing sequences 
of 1Z closed sets such that vy^ =1 B k = C, and Wg =1 B' k = C, while A^, and A' k l are 
decreasing sequences of TZ closed sets such that flg^ = B k and nfl^ = B' k . 
Both A^, and A' k l only contain conjunction and disjunction of finite formulas, thus can 
be described by PCTL^. The following proof is straightforward due to s ~ PCTL - r 
and Lemma [TT1 

(2) Suppose that ~pctl C for any n > which means that there exists s and 
r such that s ~^ r for any n > 0, but s "°pctl r - As a result there exists 
C, C and o" such that lim^oo Prob a S (C, C',i) > 0, but there does not exist a' 
such that limj_j. 0O Prob a ' tr (C, C , i) > limj-^oo Prob atS (C, C", 1). In the other word, 
liirij-Kx) Prob a ' jr (C, C", i) < lim^oo Prob atS (C, C , i) for any a' which indicates that 
there exists n > such that Prob a i <r (C,C' ,n) > Prob aiS (C,C ,n) for any a', there- 
fore s "°p CTL - r which contradicts with our assumption. 

□ 
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9. Related Work 

For Markov chains, i.e., deterministic probabilistic automata, the logic PCTL characterizes 
bisimulations, and PCTL without X operator characterizes weak bisimulations \10\ [3]. As 
pointed out in [22], probabilistic bisimulation is sound, but not complete for PCTL for 
PAs. In the literatures, various extensions of the Hennessy &; Milner [12] are considered 
for characterizing bisimulations. Larsen and Skou [T7] considered such an extension of 
Hennessy-Milner logic, which characterizes bisimulation for alternating automaton |17j . or 
labeled Markov processes [8] (PAs but with continuous state space). For probabilistic 
automata, Jonsson et al. [15] considered a two-sorted logic in the Hennessy-Milner style to 
characterize strong bisimulations. In |13j . the results are extended for characterizing also 
simulations. 

Weak bisimulation was first defined in the context of PAs by Segala [22], and then 
formulated for alternating models by Philippou et al. [19]. The seemingly very related 
work is by Desharnais et al. [8], where it is shown that PCTL* is sound and complete with 
respect to weak bisimulation for alternating automata. The key difference is the model 
they have considered is not the same as probabilistic automata considered in this paper. 
Briefly, in alternating automata, states are either nondeterministic like in transition systems, 
or stochastic like in discrete-time Markov chains. As discussed in [23], a probabilistic 
automaton can be transformed to an alternating automaton by replacing each transition 
s — > fj, by two consecutive transitions s s' and s' fj, where s' is the new inserted state. 
Surprisingly, for alternating automata, Desharnais et al. have shown that weak bisimulation 
- defined in the standard manner - characterizes PCTL* formulae. The following example 
illustrates why it works in that setting, but fails in probabilistic automata. 

Example 4. Refer to Fig. [TJ we need to add three additional states s Ml , s M2 , and s M3 in 
order to transform s and r to alternating automata. The resulting automata are shown 
in Fig. [5j Suppose that si,S2, and S3 are three absorbing states with different atomic 
propositions, so they are not (weak) bisimilar with each other, as result s^ 15 Sn 2 and s^ 3 are 
not (weak) bisimilar with each other either since they can evolve into si,S2, and S3 with 
different probabilities. Therefore s and r are not (weak) bisimilar. Let ip = P>o.4(X L(si)) A 
P>o.3(X L(s2)) A P>o.3(X L(s3)), it is not hard to see that s M2 |= (p but s^jS^ ^ </?, so 
s |= P<o(Xy?) while r ^= P<o(X(/?). If working with the probabilistic automata, s Ml , s^ 2 , and 
s^ 3 will not be considered as states, so we cannot use the above arguments for alternating 
automata anymore. 

In the definition of ~i and -<i, we choose first the downward closed set C before the 
successor distribution to be matched, which is the key for achieving our new notion of 
bisimulations and simulations. This approach was also adopted in [9] to define the priori 
e-bisimulation and simulation. It turns out that when e = 0, the priori e-bisimulation and 
simulation coincide with ~i and -<\ respectively. The priori e-bisimulation was shown to be 
sound and complete w.r.t. an extension of Hennessy-Milner logic, similarly for the priori e- 
simulation. Finally, the priori e-bisimulation was also used to define pseudo-metric between 
PAs in [91 [7]. 

10. Conclusion and Future Work 



In this paper we have introduced novel notion of bisimulations for probabilistic automata. 
They are coarser than the existing bisimulations, and most importantly, we show that they 
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Figure 5: Alternating automata. 

agree with logical equivalences induced by PCTL* and its sublogics. Even in this paper we 
have not considered actions, it is worth noting that actions can be easily added, and all the 
results relating (weak) bisimulations hold straightforwardly. On the other side, they are 
then strictly finer than the logical equivalences, because of the presence of these actions. 

As future work, we plan to study decision algorithms for our new (strong and weak) 
bisimulation and simulation relations, 
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